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INI  ERAGTION  BETWEEN  PSYCHOLOGICAL  TESTS  AND  CURRICULA 
IN  THE  MAR  r NE  HULL  REPAIRMAN  COURSE 


BACKGROUND 

Several  Army  training  courses  have  been  revised  by  changing  the 
training  methods  and,  to  a  lesser  extent,  the  course  content.  The 
method  of  training  in  the  revised  courses  places  emphasis  on  learning 
by  doing  rather  than  on  the  lecture-demonstration-performance  of 
traditional  courses.  Also,  content  of  the  revised  courses  emphasizes 
performance  of  specific  tasks  rather  than  comprehension  of  general 
principles.  One  consequence  may  be  a  less  flexible  graduate  who  needs 
retraining  as  equipment  is  modified. 

The  results  of  recent  research  on  Army  courses  showed  that  general 
aptitude,  as  measured  by  tests  of  arithmetic  reasoning  and  verbal  ability 
has  high  validity  in  predicting  success  in  virtually  all  types  of  Army 
training.  The  presence  of  this  general  predictor  suggested  that  the 
traditional  course  placed  a  high  demand  on  the  ability  to  work  with  con¬ 
cepts.  With  the  reduced  level  of  abstraction  in  the  revised  courses, 
there  may  be  less  need  for  general  learning  ability,  and  more  need  for 
specific  job-related  abilities  that  could  reflect  a  different  pattern 
of  aptitudes  for  success. 

An  ongoing  study  conducted  by  BESRL’s  Differential  Classification 
Work  Unit  is  aimed  at  gathering  evidence  about  differences  in  validity 
patterns  of  Army  classification  tests  and  some  experimental  tests  for 

courses  of  the  two  types-- traditional  and  revi sed . _ More  specifically, 

what  are  the  nature  and  magnitude  of  interaction  refects  between  train¬ 
ing  course  revisions  and  the  validity  of  tests  used\for  predicting 
success  in  the  Marine  Hull  Repairmen  Course  (44K20)?' 

METHOD 

The  criterion  measures  were  appropriate  for  the  particular  train¬ 
ing  methods  and  content  used  in  each  curriculum.  In  the  control  sample, 
which  received  the  traditional  academically  oriented  course,  the  final 
course  grade  was  heavily  weighted  by  a  multiple  choice  pencil-and-paper 
test.  In  the  experimental  sample,  which  received  a  more  performance 
oriented  course,  the  final  course  grade  was  composed  of  three  parts. 

The  first  part  was  a  general  performance  score  based  on  a  series  of 
evaluations  used  for  instruction  purposes  as  well  as  for  testing.  While 
the  man  worked  on  a  given  project,  the  instructor  corrected  his  mistakes 
and  pointed  out  areas  in  which  he  was  progressing  satisfactorily.  The 
quality  of  the  project  and  the  performance  of  the  man  were  graded  at  the 
termination  of  an  allotted  time.  The  second  part  of  the  grade  was  a 
score  on  specific  performance  based  on  a  series  of  tests  of  the  man’s 
ability  to  perform  complicated  tasks  to  rigid  specifications,  employing 
the  methods  used  in  a  major  area  of  the  course  such  as  marine  pipefitting 


No  instruction  was  given  during  the  test.  Each  step  of  the  task  was 
observed  by  an  instructor  and  graded  as  it  was  performed.  The  test  was 
terminated  and  the  project  was  graded  at  the  end  of  a  given  period  of 
time.  The  last  part  of  the  final  course  grade  was  a  written  objective 
test . 


SAMPLES  AND  PROCEDURES 

The  U.  S.  Army  Transportation  School,  Fort  Eustis,  Virginia,  pro¬ 
vided  data  on  samples  of  men  who  had  taken  the  Marine  Hull  Repairman 
course  under  the  two  curricula.  A  control  sample  consisted  of  139  men 
enrolled  in  the  Marine  Hull  Repairman  Course  in  1>67  under  the  tradi¬ 
tional  academically  oriented  training  curriculum.  The  experimental 
group  was  made  up  of  ‘>6  men  enrolled  in  the  same  course  in  1068  under 
the  revised  performance-oriented  training  curriculum.  Since  some  of 
the  training  methods  used  in  the  revised  performance-oriented  curriculum 
(experimental  sample)  were  introduced  into  the  course  on  a  gradual  basis, 
differences  between  the  control  and  experimental  samples  were  somewhat 
reduced . 

Data  on  the  control  sample  were  available  for  only  a  limited  number 
of  predictors  and  one  criterion  measure.  The  experimental  sample  had 
data  available  not  only  for  the  Army  Classification  Battery  (ACB)  tests 
but  also  for  a  number  of  other  tests  not  usually  given  to  Army  personnel. 
Table  1  displays  the  variables  for  which  data  were  available  for  each 
sample.  Data  for  five  ACB  predictors  (the  Verbal,  Arithmetic  Reasoning, 
and  Pattern  Analysis  tests  and  the  Electronics  and  Motor  Maintenance 
Aptitude  Areas)  and  one  criterion  (final  course  grade)  were  available 
for  both  samples.  The  table  also  shows  the  non-operational  variables 
included  in  the  analysis. 

A  number  of  statistical  analyses  were  performed.  A  regression 
analysis  of  the  Arithmetic  Reasoning  (AR)  test  predicting  the  final 
course  grade  was  computed  for  the  control  and  experimental  samples 
separately.  The  purpose  of  this  analysis  was  to  compare  the  regression 
of  final  course  grade  on  AR  in  the  two  training  methods,  since  the  test 
has  had  consistently  high  validity  over  the  years.  The  second  statistical 
procedure  was  to  calculate  an  intercorrelation  matrix  for  each  sample  of 
all  available  predictor  and  criterion  variables.  A  multiple  regression 
equation  was  also  calculated  for  each  sample  separately,  using  only  those 
predictors  available  for  both  samples  against  the  final  course  grade. 
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Table  I. 
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PREDICTOR  AND  CRITERION  DATA  AVAILABLE  FOR  SAMPLES* 


Data 

Experimental 

Revised 

Course 

Control 

Traditional 

Course 

Predictors 

Education 

X 

ACB  Tests 

Verbal  iVE) 

X 

X 

Arithmetic  Reasoning  (AR) 

X 

X 

Shop  Mechanics  ^SM) 

X 

Pattern  Analysis  i,  PA) 

X 

X 

Army  Clerical  Speed  Aptitude  (ACS) 

X 

Automotive  Information  (Al) 

X 

Mechanical  Aptitude  (MA) 

X 

Electrical  Information  (ELI) 

X 

Classification  Inventory  (Cl) 

X 

Army  Radio  Code  Aptitude  (ARC) 

X 

Electronics  Aptitude  Area  [(MA+2ELI ) /j] 

X 

X 

Motor  Maintenance  AA  [(MA+2Al)/5l 

X 

X 

Wechsler  Adult  Intelligence  Scale  (WAIS) 

(14  scores) 

X 

Iowa  Silent  Reading  Test  (1  score) 

X 

Prognostic  Test  of  Mechanical  Ability 

(1  score) 

X 

Domino  Test  (1  score) 

X 

Survey  of  Interpersonal  Values  (Gordon) 

(G  scores) 

X 

Criteria 

Course  Grades: 

General  Performance 

X 

Specific  Performance 

X 

Objective  Examination 

X 

Final 

X 

X 

a  "X"  indicates  data  available. 


RESULTS  AND  DISCUSSION 


In  the  experimental  sample,  the  specific  performance  score  had 
extremely  high  correlation  with  final  course  grade  (r  =  .  )2 ,  Table  2); 
the  general  performance  score  and  the  objective  test  had  lower  coeffi¬ 
cients  (.'/'4  and. .42.  respectively).  It  would  appear,  then,  that  a  man's 
standing  in  his  class  was  fairly  well  defined  by  his  ability  to  perform 
the  specific  tasks.  Conversely,  the  objective  test  carried  only  the 
weight  due  to  the  part-whole  correlation  reflecting  its  presence  in  the 
final  course  grade,  desp  te  its  high  standard  deviation. 

Table  2 


MEANS,  STANDARD  DEVIATIONS,  AND  INTERCORRELATIONS 
OF  CRITERION  SCORES  FOR  THE  EXPERIMENTAL  SAMPLE 

(N  =  >56) 


Criterion 

Mean 

S.D. 

Intercorrelat ions 

Final  Course  Grade 

4 

FCG 

General  Performance 

86 

4 

.74 

GP 

Specific  Performance 

-1 

6 

.  12 

.54  SP 

Objective  Test 

80 

12 

.42 

.38  .26  OBJ 

Table  3  shows  all  the 

valitidy  coefficients 

obtained  in  both  samples 

In  the  experimental  sample 

,  the  cognitive 

predictors  attained  highest 

validity  with  the  general 

performance  rating  and 

lowest  validity  with  the 

specific  performance  ratings.  Coefficients  are  shown  for  measures  out¬ 
side  the  ACB  and  for  education  simply  as  a  matter  of  record,  inasmuch  as 
the  data  were  analyzed. 

The  highest  validity  coefficient  against  the  specific  performance 
score  (r  =  .43)  and  against  final  course  grade  (r  =  .4>)  in  the  experi¬ 
mental  sample  was  attained  by  the  only  noncognitive  test  in  the  ACB,  the 
Classification  Inventory  (Cl).  The  Cl  is  a  self-description  inventory 
keyed  to  identify  men  who  are  well-adjusted  and  cooperative  in  their  atti¬ 
tudes  toward  rigorous  Army  training  and  Army  life.  Individuals  in  the 
experimental  sample  who  had  a  high  Cl  score  may  also  have  received  a  high 
performance  score  because  of  their  close  cooperation  with  the  instructor 
and  performance  rater.  Two  scales  in  Gordon's  Survey  of  Interpersonal 
Values.  Benevolence  and  Leadership,  indicated  some  support  for  the  Cl 
validity  <r=.4j).  The  Benevolence  Scale,  which  measures  traits  opposite 
to  those  of  the  Cl  (r  =-.21)  and  the  Leadership  Scale  (r  =-.47)  had  a 
moderate  negative  correlation  with  final  course  grade  (r  =-.37)>  The 
Leadership  Scale  had  moderate  positive  correlation  with  the  Cl  (r  ■  .29) 
and  with  final  course  grade  (r  =  .36). 
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Table 


VALIDITY  COEFFICIENTS  FOR 

EXPERIMENTAL 

AND  CONTROL 

SAMPLES* 

Predictors 

Cri ter ia 

Final  Course 
Grade 

Gen 

Perf 

Score 

Specific 

Perf 

Score 

Obj 

Test 

Exp 

Con 

Exp 

Exp 

Exp 

ACB 

Verbal 

A • 

.4  A 

/  V 

57 

Arithmetic  Reasoning 

40 

50 

7  r 

J  J 

Shop  Mechanics 

AO 

•'  V 

1 

Or. 
c-  ) 

40 

Pattern  Analysis 

37 

57' 

GP 

25 

7  7 

Army  Clerical  Speed  Aptitude 

51 

50 

27 

25 

Automotive  Information 

y, 

54 

50 

55 

Mechanical  Aptitude 

4  5 

26 

47 

Electronic  Information 

54 

45 

25 

44 

General  Information 

4') 

>'"• 

54 

65 

Classification  Inventory 

4  • 

71 

45 

1  ? 

Army  Radio  Code  Aptitude 

27 

40 

17 

41 

Electronics  Aptitude  Area 

5  • 

a 

lx0 

27 

4  *■ 

Motor  Maintenance  Aptitude  Area 

45 

r;  , 

r)  • 

50 

40 

WAIS 

Verbal 

4  i 

’)  * 

Information 

51 

55 

27 

Ss 

Comprehension 

4V 

4  4 

57 

5” 

Arithmetic 

51 

47 

20 

55 

Similarit ies 

4  • 

rr.>4 

•2  7 

58 

Digit  Span 

52 

•2, 

26 

2 

Vocabulary 

^  •; 

4  ■ 

26 

71 

Performance 

56 

61 

25 

38 

Digit  Symbol 

14 

58 

-05 

1  ■ 

Picture  Completion 

V,-2, 

76 

26 

50 

Block  Design 

45 

CO 

52 

3C 

Picture  Arrangement 

2- 

42 

25 

21 

Object  Assembly 

12 

27 

0  ■ 

2"' 

Full 

44 

65 

Jp 

70 

Ml  SC . 

Iowa  Silent  Reading 

X  -< 

57 

2  * 

5‘- 

Mechanical  Abilities 

42 

67 

26 

36 

Domino 

7  ) 

6  3 

2  • 

Educat ion 

27 

1? 

21 

Gordon  SIV 

Support 

-  0  ’ 

-17 

14 

Conformity 

-10 

-2  1 

-07 

-24 

Recognition 

00 

-07 

01 

03 

Independence 

1.0 

p  r: 

-0V 

23 

Benevolence 

-57 

-40 

-20 

-42 

Leadership 

_ 

24 

‘:0 

16 

‘Decimal  points  omitted. 
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A  comparison  oL'  the  control  ami  experimental  samples  with  respect 
to  the  repression  of  final  course  grade  on  All  appears  in  Figure  1 .  The 
mean  final  course  grade  in  each  Alt  interval  is  indicated  in  the  AR 
column  for  each  sample.  Trend  lines  in  both  samples  showed  a  clear 
linear  relationship  with  nearly  identical  slopes,  indicating  that  the  AR 
test  functioned  in  both  samples  equally  well  as  a  predictor  throughout 
tlie  score  range.  Differences  between  the  trend  lines  indicated  either 
better  performance  or  relaxed  standards  in  the  experimental  course. 

Zero-order  correlation  and  multiple  regression  analyses  were  per¬ 
formed  using  five  predictors  against  the  final  course  grade  for  each 
sample.  The  predictors  were  the  Verbal  (VE).  Arithmetic  Reasoning  (AR), 
and  Pattern  Analysis  PA  1  tests,  plus  the  Electronics  <  EL)  and  Motor 
Maintenance  MM'  Aptitude  Area  composites.  Data  for  these  variables  are 
summarized  in  Table  i . 

Within  the  experimental  sample,  the  zero-order  validity  coefficients 
for  the  five  predictors  shown  in  Table  4  were  of  the  same  general  magni¬ 
tude.  with  MM  the  highest  (r  =  .4'))  and  PA  the  lowest  (r  =  -j().  The  beta 
weights  in  the  experimental  sample  also  indicated  a  stronger  contribution 
of  MM  to  the  prediction  of  final  course  grade,  as  would  be  expected  in  a 
course  emphasizing  more  practical  mechanics.  The  validity  coefficients 
were  uniformly  lower  in  the  experimental  sample  than  in  the  control  sample, 
perhaps  as  a  result  of  differences  in  the  criterion  measures.  Within  the 
control  sample,  the  validity  coefficients  were  more  varied.  MM  fr  =  .5 "• ) 
and  EL  r  =  .61 )  were  the  best  of  the  five  predictors  even  though  the 
curriculum  for  the  control  sample  was  more  academically  oriented.  PA, 
again,  had  the  lowest  of  the  five  (r  =  .yj) .  The  beta  weights  for  EL  and 
MM  in  the  control  sample  also  pointed  to  a  higher  contribution  of  the 
mechanical  predictors. 

All  five  predictors  showed  significant  validity  in  both  samples. 

In  terms  of  beta  weights,  the  MM  aptitude  area  and  the  AR  test  showed 
high  independent  contributions  in  both  samples.  VE  was  also  significant 
in  the  experimental  sample  and  EL  carried  the  highest  weight  in  the 
control  sample.  The  multiple  R  of  .4~  in  the  experimental  sample  was  not 
signi f i cal  1 y  different  at  the  .'V\  level  from  the  multiple  R  of  .66  in  the 
control  sample.  The  relatively  high  negative  beta  weight  of  -.12  for  VE 
in  the  control  sample  may  be  peculiar  to  the  sample.  The  data,  then,  did 
not  indicate  that  the  reduced  level  of  abstraction  in  the  revised  course 
required  a  markedly  different  pattern  of  aptitudes  for  success. 
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AR  TEST  SCORES 


Figure  1.  Regression  of  final  course  grade  on  Arithmetic  Reasoning  test  scores  in  the 
Marine  Hull  Repairman  Course 


Table  -1 


COMPARISON  OF  EXPERIMENTAL  AND  CONTROL  SAMPLES 
ON  FIVE  PREDICTORS  AND  ONE  CRITERION 


Predictors 

Mean 

S.D. 

Val id ity 
Coefficient  s 

Multiple 

R 

Beta 

Weight 

Expa 

Con8 

Exp 

Con 

Exp 

Con 

Exp  Con 

Exp 

Con 

VE 

100 

24 

24 

.40 

.44 

.47  '  <637 

.14 

-.12 

AR 

100 

•7 

po 

20 

.40 

.33 

•15 

.24 

PA 

103 

106 

24 

1  • 

•  37 

•  37 

.01 

-.02 

ELb 

104 

103 

20 

20 

.30 

.61 

.01 

•35 

Mtf 

108 

103 

1  * 

1  ■ 

•  43 

•25 

•25 

Criterion 

FCG 

8p 

81 

4.0 

3.3 

‘Experimental  sample  size  =  36 ;  Control  sample  size  =131. 
b EL  Aptitude  Area  score  =  ('MA  f  2ELl)/3;  MM  Aptitude  Area  score  = 
( MA  +  2AI)/?. 


SUMMARY 

The  training  methods  and  course  content  in  the  Marine  Hull  Repairman 
Course  have  been  revised  by  placing  greater  emphasis  on  learning  through 
actual  performance  of  specific  tasks  than  on  learning  through  presentation 
of  the  more  abstract  principles  linked  to  the  tasks.  In  order  to  obtain 
an  indication  of  the  nature  and  magnitude  of  the  interaction  effect  between 
training  methods  and  tests,  the  validity  patterns  of  tests  for  a  sample  of 
men  (N  =  13 r* )  who  took  the  previously  standard  acdemically  oriented  course 
were  compared  with  the  validity  patterns  of  tests  for  a  sample  of  men 
(N  =  36)  who  took  the  revised  performance  oriented  course.  While  for  the 
revised  course  sample  the  full  ACB  and  a  number  of  other  test  scores  were 
available  as  predictors,  the  standard  course  sample  had  only  five  scores 
whose  validities  could  be  compared  across  the  samples. 

Zero-order  correlation  and  multiple  regression  analyses  of  the  five 
common  predictors  against  the  final  course  grades  did  not  uncover  signifi¬ 
cant  differences  in  validity  patterns  between  the  two  samples  to  support 
the  hypothesis  of  greater  relative  validity  in  the  control  sample  for  the 
academic  predictor  tests  (VE  and  AR),  or  greater  relative  validity  in  the 
experimental  sample  for  the  mechanically  oriented  aptitude  areas  (EL  and 
MM).-  The  highest  validity  coefficient  against  the  specific  performance 


-  8  - 


Mi 


rating  and  the  final  course  grade  in  Lhe  experimental  sample  was  not  one 
obtained  through  a  test  of  mechanical  ibilities  but  was  attained  by  the 
only  noncognitive  test  in  the  ACB,  the  Classification  Inventory  (Cl). 

If  the  high  validity  for  Cl  proves  to  be  stable  in  subsequent  studies, 
a  new  dimension  in  the  criterion  for  the  revised  course  will  have  been 
indicated. 


